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Abstract 

Online courses are appearing at a high rate, increasing the competitiveness of the distance 
learning market. Reluctance to invest in this area is due to cost and quality concerns. This study 
reports the findings of a survey of 700 professional and graduate education students regarding 
their comfort with technology, satisfaction with those experiences, and perceived quality, given 
their experience with online or hybrid courses. A structural equation model yielded strong fit to 
the data. For students with Online course experience, comfort with technology was related to 
satisfaction with online course experience which was related to perceived quality; motivation to 
learn more about technology was also related to satisfaction of online learning experience. For 
students with hybrid online course experience (WebCT), comfort was related to satisfaction and 
motivation; motivation was also related to satisfaction; and satisfaction was ultimately related to 
perceived quality. In this group (WebCT), comfort was also directly related to perceived quality. 
Finally, among students with no online course-related experiences, comfort was related to 
motivation to learn about technology, but neither of these factors were related to perceived 
quality of online courses. For students with limited or no online course experience, perceptions 
of online course quality were more difficult to explain. 
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Introduction 

Technological advances since the 1990s have led to the increasing integration of web- 
based and web-enhanced resources into instructional practices. It is difficult to find a higher 
education course that does not employ or take advantage of technology in some way. At one 
extreme, the number of online courses offered by different colleges and universities is growing at 
an astonishing rate. During the 2000-2001 academic year, 56% of all 2-year and 4-year degree 
granting institution offered online courses while an additional 12% were planning on offering 
online courses within the following three years; this included 127,400 online courses attracting 
an enrollment of over three million (NCES, 2003). 

Colleges and universities have been investing a great deal of effort and resources to 
modify existing face-to-face courses to meet the specific characteristics of online learning 
environments. Financial costs have been the largest source of reluctance: 26% of institutions 
reported that program development costs kept them from starting or expanding online course 
offerings. For the 31% of institutions that did not offer online courses, reasons included lack of 
fit with mission (44%), development costs (33%), concerns about course quality (26%), limited 
technological infrastructure (24%), and lack of perceived need (22%) (NCES, 2003). 

Higher education institutions face a tremendous challenge of designing effective and 
sustainable online courses. Numerous studies have attempted to compare the quality of online 
versus traditional, face-to-face instruction. The meta-analysis conducted in 2004 by Bernard, 
Abramy, Fou, Borokhovski, Wade, Wozney, Wallet, Fiset, and Huang, focused on student 
achievement, attitudes, and retention. The results indicated small significant effects favoring 
distance education on all three measures; yet effect sizes of all measures varied a great deal 
which calls for a careful interpretation of these findings. 
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There has been extensive discussion to identify the key components of successful 
instructional design to increase the outcomes of online learning (Gunawardena, Lowe, & 
Carabajal, 2001; Kreijns, Kirschner, & Jochems, 2003; Mallen, Day, & Green, 2003). 
Researchers have created specific evaluation models to measure the outcomes of online learning 
(Jones & Paolucci, 2000; Gunawardena, Carabajal, & Lowe, 2001). Nonetheless, the success of 
online courses may rest in the perceptions of the student. And as with any product, educational or 
otherwise, perception in an open marketplace, much like the educational course-taking 
marketplace, may ultimately determine whether students continue to enroll in online courses. 
Purpose 

The focus of this paper is the identification of variables that may impact the degree of 
students’ satisfaction with online learning experiences and learning outcomes or perceived 
quality. These variables could be considered during an online course instructional design and 
instruction to promote high quality online course development and marketing. Early 
identification of relevant variables could be used to increase the likelihood of positive online 
learning experiences. For example, if a limited level of technical skills jeopardizes attainment of 
positive learning experiences, students would be referred to the appropriate resources to gain the 
essential technical skills before they enroll for an online course. 

The model includes the following variables: 

1 . Level of comfort with technology, including various online -related computer operations; 

2. Motivation to learn technology tools; 

3. Level of satisfaction with previous online learning experiences; and 

4. Perceived quality of online courses. 
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The model includes multiple outcomes and multiple moderators. We assessed their 
interrelationships with a structural equation model to provide a more complete picture (described 
below). Primary outcomes included satisfaction with the learning experience and perceived 
quality of the learning experience. Finally, there are three relevant groups of students for 
comparison purposes, including (a) students with online course experience, (b) students with 
hybrid course experience (typically in the form of courses using WebCT, an online course- 
support tool), and (c) students with no online-related course experience. 

Perspectives 

Engaging in an online learning process requires a certain level of comfort with computer 
and web technologies (Lee & Witta, 2001). In order to fully participate in online instructional 
activities, to access instructional materials, and to interact with an instructor and peers, students 
need to be able to use a variety of modern hardware and software applications. Online learners 
lacking the required technical skills are at risk of suffering from computer anxiety (Loyd & 
Gressard, 1984). Those students must invest extra effort in learning the necessary technical skills 
while being expected to master new course content. 

There has been some controversy regarding the relationship between the level of comfort 
of using the Internet and the degree of student satisfaction with online courses. DeBourgh (1999) 
studied student satisfaction with online learning and reported that being at ease with the World 
Wide Web was not a predictor variable for satisfaction with online learning. This finding was 
confirmed by Westbrook (1999). However, Stokes (2003) found a significant relationship 
between the two variables; students who felt more at ease using Internet were more likely to be 
satisfied with their online learning experiences, than those students who did not feel comfortable 
using internet. 
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Similarly, Wright (1999) found that the students taking an online course were more 
satisfied with their learning experiences than students enrolled in a similar course delivered in a 
face-to-face format. In addition, St. Pierre and Olson (1991) identified student willingness to take 
an online course again and their willingness to recommend an online course to a friend as the 
best measures of student satisfaction. 

Stokes (2003) concluded that satisfaction with online learning was not influenced by the 
quantity of previous online learning experiences; yet there is a significant difference between 
more- and less-experienced users in this context. However, Wagner, Holloway, and Garton 
(1999) suggested that there might be a connection between a student’s first online learning 
experiences and their satisfaction. This area of inquiry is quickly becoming complex. 

Methods 

A survey was developed, reviewed by instructional technology experts and researchers, 
and piloted with a sample of the target audience prior to publication online. This survey was 
conduced online. Pre-survey e-mail notices about the survey were sent to the approximately 3000 
enrolled students in a post-BA, professional, and graduate school of education. The survey was 
made available via the Internet for four weeks. During that time, students were invited to 
complete the survey through an initial email messages and one reminder two weeks later, both of 
which contained the website address of the survey. This resulted in 721 completed surveys for a 
24% response rate. We expect that the functional response rate was likely over 50% as many of 
the 3000 students were inactive or no longer attending (there was no way to easily remove these 
students without additional authorization). In all, 698 surveys were usable in this project, as 23 
contained significant missing responses. 
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Survey responses were factor scored (based on a principal-axis factor analysis) to create 
the comfort with technology scale. Scores for other variables were used as reported on the 
survey. All responses were submitted to a structural equation model employing AMOS 4.0 
(Arbuckle, 1999). Other descriptive analyses were completed using SPSS. 

Results 

Among the 698 students with complete surveys, 39% were 16 to 25 years old, 38% were 
26 to 35, 13% were 36 to 45, and 10% were 46 or older. There were no statistical significant age 
differences between those students with online experience and those without. 

Most students felt comfortable with basic computer-based technology activities, such as 
using email, typing, accessing the Web, sending, receiving, and downloading documents. Fewer 
students felt comfortable with more advanced activities, including downloading multimedia 
materials, listening to audio or viewing video on the computer (see Table 1). 



Table 1 

Comfort Level with Computer-Based Technology Activities 





Mostly 


Somewhat 


Somewhat 


Mostly 




uncomfortable 


uncomfortable 


comfortable 


comfortable 


Using email 


5% 


0% 


2% 


92% 


Typing, key boarding 


6% 


1% 


6% 


87% 


Accessing the Web 


6% 


1% 


6% 


88% 


Sending documents 
electronically 


8% 


8% 


21% 


63% 


Receiving documents 
electronically 


7% 


5% 


22% 


66% 


Downloading documents 


8% 


10% 


27% 


55% 


Downloading multimedia 
materials 


15% 


23% 


34% 


29% 


Listening to audio on the 
computer 


13% 


15% 


28% 


44% 


Viewing video on the 
computer 


17% 


19% 


31% 


33% 
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Comfort with technology, as described above, was not related to online course taking 
behavior; students with online course experience, hybrid course experience, or no online course 
experience reported the same level and range of comfort (F[2,695]=0.01,p=.99). In addition, 
there was a nonsignificant correlation between comfort and number on online courses taken 
(r=.08, p=.30), which further suggests no relationship between comfort with technology and 
online course taking behavior. 

Students reported high levels of motivation to learn more about using computer-based 
technology; 31% reported excellent motivation, 44% good motivation, 22% fair motivation, 3% 
poor motivation. There were small correlations between level of comfort and motivation to learn 
with the more advanced activities (Spearman rho ~ .25 to .28). There was a weak correlation 
between motivation to learn and age (Spearman rho ~ .19). All Spearman correlations between 
age and comfort levels with computer-based technology activities were less than .15. 

Nearly three-fourths of all students (73%) had some experience with WebCT (Web-based 
course support tools). Of these students, the perceived helpfulness of WebCT for learning was 
not unanimous; 14% reported that WebCT was very helpful, 39% moderately helpful, 36% a 
little helpful, and 11% not helpful at all. 

Fewer students, nearly one-fourth (24%), had experience with online courses. Of those 
who had taken online courses, 22% had taken more than one. Satisfaction with online courses 



was relatively high (Table 2). 
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Table 2 

Satisfaction with Online Courses (n=168) 



Percent 


Completely dissatisfied 


1.8 


Mostly dissatisfied 


8.9 


Somewhat dissatisfied 


11.9 


Somewhat satisfied 


24.4 


Mostly satisfied 


39.9 


Completely satisfied 


13.1 



Students reported to like several aspects of online courses (Table 3). Mostly, they 
appreciated the flexibility of study time (48%) and less need to go to campus (24%). On the other 
hand, students least liked the limited face-to-face interaction (49%). 



Table 3 

What do you like most or least about taking Online Courses? (Percentages Reporting) 





With online course 
experience 


Without online course 
experience 


Like Most 


Like Least 


Like Most 


Like Least 


The online delivery methods 


1.8 


11.4 


3.0 


7.0 


Limited face-to-face interaction 


3.0 


49.1 


0.6 


67.9 


Flexibility of study location 


14.9 


0.6 


19.7 


0 


Reliance on my own self-discipline 


4.8 


12.6 


3.4 


14.6 


Flexibility of study time 


47.6 


0.6 


31.7 


0 


Spending less time on the computer 


0 


9.0 


0.6 


6.1 


Less need to go to campus 


23.8 


0.6 


38.5 


0.2 



When asked to imagine taking an online course, students without online course 
experience reported to imagine liking the fact that there would be less need to go to campus 
(39%), flexibility of study time (23%, half compared to students with online course experience 
[48%]), and flexibility of study location (20%, similar to those with online course experience). 
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On the other hand, students reported to imagine not liking the limited face-to-face interaction 
(68%) at a higher rate than student with online course experience (49%) and being reliant on 
their own self-discipline (13%, similar to those with online course experience; see Table 3). 

Among students who had online course experience, half (52%) reported to agree that 
online courses meet the same quality standards as classroom courses (Table 4). Even so, 89% 
reported that they would consider registering for another online course; 11% would not. 

Again, when asked to imagine taking an online course, fewer students (about 21%) with 
limited (hybrid) or no online course experience reported to agree that online courses would meet 
the same quality standards as classroom courses compared to student with online course 
experience (52% agreeing; see Table 4). 



Table 4 



Online Courses Meet the same Quality Standards as Classroom Courses 





With Online 
course experience 


With hybrid (WebCT) 
course experience 


With NO online 
course experience 


Disagree 


111 


38.5 


33.1 


Tend to disagree 


22.8 


34.6 


36.6 


Tend to agree 


33.5 


15.6 


18.0 


Agree 


18.0 


5.6 


4.1 


Do not know 


3.6 


5.6 


8.1 


n 


168 


358 


172 



Structural Equation Models 

As an initial set of analyses to understand the relations among the primary explanatory 
variables, we found independence between comfort with basic computer operations and 
advanced computer operations as well as between comfort with technology (basic and advanced) 
and motivation to learn about technological tools. In our sample, 70% of students had 
experienced hybrid courses, where some component of the course was delivered online (typically 
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an online discussion board, online quizzes, and online assignment submission). In all 22% had 
taken online courses, ranging between 1 online course (77%) to 4 or more online courses (4%). 

Structural models were examined for three groups. Group one (WebCT; n=358) was 
comprised of students who had experience using WebCT and had not previously taken an online 
course. Group two (Online; n=168) were the students who had previously taken on online course. 
The third group (None; n=172) had neither WebCT nor online course experience. The following 
is a comparative description of the fit of the structural model and path coefficients for the three 
groups. 

To determine the degree of fit, absolute, relative and parsimonious fit were assessed. To 
determine absolute fit, the chi-square (y^) goodness of fit test was used. Because chi-square 
statistic is dependent on sample size, two additional indices were also examined (Floyd & 
Widaman, 1995; Reise, Widaman, & Pugh, 1993). The Goodness of Fit (GFI), the Adjusted 
Goodness of Fit (AGFI) (Joreskog & Sorbom, 1989), and the Root Mean Square Error 
Approximation (RMSEA; Steiger & Eind, 1980) indices were also examined. The GEI and 
AGEI indices range between 0 and 1 and .90 is a suggested acceptable value. An RMSEA value 
of .05 or less indicates a close fit to the data; .05-. 10 a moderate fit; and above .10 is a poor fit 
(Browne & Cudeck, 1993). Overall the model fit was good for all groups. Table 5 is a summary 
of the indices of fit for the three groups. 

Eigure 1 represents the structural equation model for the Online group. Noteworthy 
features of this model include the positive relationship between comfort and satisfaction as 
illustrated by the statistically significant standardized path coefficient (.15). The relationship 
between satisfaction and quality is also positive, significant, and much stronger (.51). Taken 
together this indicates that students who are more comfortable with technology are also more 
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likely to be satisfied with online courses. This, in turn predicts higher levels of perceived quality 
with online courses. There is not a direct relationship between comfort with technology and 
perceived quality of online courses (effect is nonsignificant). There is an indirect path from 
motivation to satisfaction (direct effect of .16) and finally to perceived quality (indirect effect of 
motivation on quality = .16 x .51 = .08). All paths were positive and significant indicating that 
students with higher levels of motivation are more likely to be satisfied with online courses and 
will ultimately have more positive perceptions of quality of online courses. 




Figure 1 . SEM model with standardized path coefficients for students with online course 
experience (Online). 

Figure 2 is a representation of the structural equation for the WebCT group. Significant 
paths were noted from comfort to motivation (.26), motivation to satisfaction (.14) and 
satisfaction to quality (.12). There was a positive, significant path from comfort to quality in this 
group. In this group, satisfaction was not predicted by comfort. 
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Figure 2. SEM model with standardized path coefficients for students with hybrid-online course 
experience (WebCT). 

The model for the group with no experience was simplified (Figure 3) because no 
satisfaction variable was available with this group. In this model the path from comfort to 
motivation was positive and significant (.38). Comfort did not predict perceived quality for the 
no experience group, nor did motivation predict quality. For students with no online-related 
course experience, perceptions of quality remain unexplained by comfort with technology and 
motivation to learn about technology. 
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Figure 3. SEM model with standardized path coefficients for students with no online course 
experience (None). 

Table 6 can be used to compare standardized path coefficients between groups. Comfort 
was a significant predictor of perceived quality for the WebCT group but not for the Online 
course group. The strength of the relationship between satisfaction and quality is nearly four 
times higher for the Online group than the WebCT group. These coefficients indicate that for 
students who have never taken an online course, comfort with technology is predictive of 
motivation to learn more about online learning tools. When students have previously taken on 
online course, the relationship is not significant. For these students, comfort is significantly 
related to satisfaction with online learning and in turn is predictive of a positive perception of 
quality of online courses. 
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Table 5 
Fit Statistics 



Fit Statistic 




Group 




Online 


WebCT 


None 


GFI 


.999 


.998 


1.00 


AGFl 


.995 


.982 


.826 


CFI 


1.00 


.993 


1.00 


RMSEA 


.000 


.029 


.213 


Chi-Square, df, p 


.166, l,p=.68 


1.303, l,p=.25 


.000, 0 



Table 6 

Standardized Path Coefficients 


Paths 




Group 






Online 


WebCT 


None 


Comfort Satisfaction 


* 

.15 


.003 


- 


Comfort -> Quality 


-.08 


.16 


.04 


Comfort Motivation 


.12 


.26* 


.38* 


Motivation Satisfaction 


.16* 


.14 


- 


Satisfaction Quality 


.51* 


.12 


- 


Motivation Quality 


- 


- 


.00 



* p<.05 



Discussion 

With models like these, measurement questions prevail: Are we measuring the right 
constructs? Are we measuring the constructs correctly? Only through replication and 
modification can we begin to answer these questions. This study advances this simple goal. 

Comfort with computer operations and online-technology tools does not appear to be 
related to online course taking behavior (number of online courses taken). This finding counters 
common expectations: comfort with technology does not limit who enrolls for online courses. 
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However, we also found a negative relationship between the number of online courses 
taken and satisfaction. In all, 23% of the students who had online course experience were 
dissatisfied at some level, whereas 47% reported that the online components of hybrid courses 
(WebCT) were less than helpful. This indicates an important area for study — satisfaction with 
online learning experiences may prevail as a determining factor for the success of online courses 
in the future. 

This is even more important with the finding of the relation between satisfaction and 
perceived quality of online courses. For students with online course experience, satisfaction with 
their experiences was a much stronger predictor of perceived quality of the courses than comfort 
with technology or motivation to learn technology; whereas both comfort and motivation were 
related to satisfaction. 

For students with only hybrid course experience (WebCT), comfort was not related to 
satisfaction with those courses and more strongly related to motivation, which in turn was related 
to satisfaction. For this group, comfort was slightly more related to perceived quality of online 
courses than was satisfaction with their hybrid course experience. 

Finally, for students with no online or hybrid course experience, comfort was strongly 
related to motivation to learn, whereas neither were related to perceived quality. For student with 
limited or no online course experience, perceptions of online-course quality were harder to 
predict. For those with online experience, satisfaction with such experiences was the overriding 
predictor. Recall from Table 3, that for students with online course experience, the most liked 
aspects included flexibility of study time, study location, and less need to go to campus. These 
were also the expected favorite aspects of students with no online course experience. In addition, 
both groups reported to be less favorable about the limited face-to-face interaction, but those 
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with no online course experience were much more unfavorable (68%) about this aspect of online 
courses compared to students with online course experience (49%). 

In today’s technologically oriented, information dependent economy, the availability of 
online courses will continue to increase. The success of higher education institutions entering the 
distance learning market will depend on greater understanding of the contexts investigated here. 
Further research in this area will also help target deeper research as we begin to understand 
components of perceived and realized satisfaction and quality of online learning experiences. 
Being able to understand the needs of students, support students in online courses, and promote 
successful learning experiences will be critical in the overall success of the online learning arena. 
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